I am trying to reproduced result from the paper "DYNAMIC COATTENTION NETWORKS FOR QUESTION ANSWERING" (https://arxiv.org/abs/1611.01604). I have implemented the code in pytorch but it is overfitting. In the paper it is mention that the authors use dropout for regularization. I added dropout and it helps a bit but not too much. I am also curious if it is possible to get feedback on my model code.

Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.